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Ai^ the accouiitabili ty lOveMent: has brought forth 
, renewed interest in the guality pf teaching and new deia^^s for - 
faculty fevaluation, it could be expected that the relatio/iship 
between the two would have created /the foundation for a rigorous 
prograi of faculty evaluation. educators continued a 

fruitless search for qualities aftl^oehaviors thit defined the perfect 
teacher. ^ rn-the ■eantiie^ we kave fost out perspective €*out the r 
function of faculty and the .real porpose of higher education,L and 

^^iiilarly, at)out the purpose, of faculty evaluation. Evaluation is not 
ajQ end in, itself ; it is the Means to itproving instruction, to 
providing a better education for our students. It lust be recogniaed 
that higher education is accountable first to the students for high 
quality instruc^tion. There is a vatiety of sources of evaluative 
d^ta/ each source having certain lititations. In the OCIJl School of 
i^ntistry.an ongoing, systen^tic, and cbaprehensive * program of 
teaching ^and curricular evaluation has been established that uses the 
• criterion-referenced ^ystei of instruction and- ■easureMent# 
supplgiented inf orsaAy by other,- ftcre traditional Measures. An 

' itistractionai developient pirograi £% faculty is an integt^l part of 
ihe evaluation systei. Making available the Icind of help that will 
€inable thea to BaxiMize attainient of their x^ntracted 
responsibility. Onle^s colleges and universities today lake siiilarly 
serious coMitients to iMpjroving instriction and idvaluatihg teaching 
by providing substantial financial aivd personnel support, higher 
'Education may never r^over its forier .importance and respect in 

, society. (iuthor/MSft) \ « 



■ Documents acquired by ERIC include many informal unpubUshed materials not available from other sources. ERIC makes every 

effort to obtain the best copy available. Nevertheless,- items of marginal reproducibility are often encountered and this affects the 
• S/- ""CToflche ^nd hardcopy reproductions ERIC makes available via the ERIC Document Reproduction Service (EDRS). 

. ^ IS not responsible for the quality of the original document Reproductions supplied by EDRS are fhe best that can be made fronf 



7 



\ ■ - 

'..^ACOLTY "EVALUATION IN AM /(CCOUtJTABLE WORLD: 
• ° HOW DO YOU DO LT?* * 55|2 ^^l^s'l 

0 Clare Rosex ' 

NaJ Evaluation and Traininr Institute , -^^J^sS 

Every -year vast • numbers, o r faculty members, are ''evaluated", 
and major tiecisions af fectinp; their lives and- the lives of their 
' . stud^ts are made on th^ basis of Ihac^unates unreliable, infer^n- 

1 tial, subjective and unsysteWatically, collected information. Oft 
' paper, particularly at larrge universities, facul^ty' are evaluated 

in th\ree arefis research , ""community service and teaching. In ^ 
reality, we all 'knov; that I'pubrish or -perish'* prevails in Xhe trln- 
i*ty. We hear that th^ majority of administrators at liberal arts 
colleges are themselves* satisfied and bgl'leve that' their faculty 
are satisfied with *tKe policies, and practices used for evaluation. 
In fact, we krfovr tl^at this is not true. The majority of^ faculty 
evaluations* at liberal arts colleges are based on chairmen and 
d^ans^ subjective evaluations, and faculty look upon them much as 



a chi^Ld forced to take cod liver oil' — with suspicion and dis- ' 
taste.* ; , 

If the audience detects a not^ of dissatisfaction in these 
statements, they are correct. I am appalled at the shoddy j>rao-- 
ti<je^ that charac'C^hj;^-ze the- '"state-of-the-art" . I am dismayed by 




the Cact tTiat many of the very same critics of faculty eval^ua- 
\j tion who argue that teaching cannot be assessed- because it 
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is too complex a process dricl that "really important learning" 

Acannot be measured (an idea that survives without a shred of 
# ^ 

evidence), then accept evaluation procedures based on, a generally 
haphazard collection of opinions from students, colleagues and 
adminis^ators ; and everji -oaore , accept almost umiritlcally Important 
decisions based on Tnferences drawn from these opinions. 

m 

^ Certainly, I am not alone in my cdVicern 'about faculty evalua- 

tion*. Several of education's formidable gadflys have been arguing 
for systematic faculty . e.valuatioa procedures for oVer thirty years. 

,In my mind, however, the progress we have made would astonish even 

the most cautious tortoise. During the last few years, as ^e 

accountability movement brought forth renewed Interest in th^ quality 

of ,teaching and new demanijs fof faculty: evaluation, one would have 

thought that the rather obvious relationship between the two factors^ 

would ha^^B emergecj as the foundation for a rigorous program of fa- 

I ^ 

. culty evaluation. No such logic-'prevailed; and people continued 
t^lr fruitless search for qualities and behaviors th^t defilfe the 
perfect teacher. Hundi^eds of studies were designed to isolate an 
indfx. or .instructional skills, and almost without exception, all 
failed to yield even a hint. ^ ^ ^ 

In the meantime, v;e somehow lost our perspective fj-bout thte 
function of our facult/ and the real purpose of higher education. 
We have lost our perspective about the dual purposes of faculty 
evaluation: 1) to render Judp:ments about an individual teacher's- 
instructional effectiveness as part of hi^ competence at a faculty 
member; but, even' more important, ?) to provide faculty members 



with the means to improve their teachjnr. Facufty evaluation 

is not an efid in itself.^ It is the means to improvinr; instrucj^ion, 

to providinp; 'a better education for our s^uden^^s. 

/ I-et us suppose for a moment that a crollece or university was 
r.enuinely Interested In establishinr a Judicious and effective/ • 

Taculty evaluatioYi system. V/hat should they do? Conrruent^ witli / 
the title of thjfs symposium, ^'Faculty evaluation in an accountable 
world: How do you do it?'*, we must first ask to v/h6m is hip:her 

'education accountable , . . an-i for v;hat? For my part, there is 
little doubt. We are aocountnbO e- to the students I ,And for what? ' 
Wiquestionably , hich quality instruction. 

If v;e accept that a htrli nuality .education is* the basic ' mis- 

,s4on of higher- ec^ucatien^ then the procedures usec^ for evaluating 
faculty should fa t lea st incJude soinp evidence of a facalt'y member's 
ability to promote learninn- learr.inr is the foal , tlfien a 

crucial element in the^ teacher eva luat 1 r,n ^^rooess is'the'means 
v/hereby both student and teicher can dote^mine whether the atudent 
has In fact learned what he was supposed 'to have learned. Can the 
student analyze the problem, formulate hypotheses,, solve the equa- 
tion, translate the sentence, describe ,the Imapery? Doeb the stu- 
dent derionnlrnte awarener,r> and senj^i 1 1 vlty , us,e critical think- 
Inr, Jov^ l^arnlnr ^nr' l^^^rnlnr'^s r^^he? Tf not* why not? What are' 

tlie errors and v;hy dr> they ex-j^^t? The annwers to th^se questions 

I 

provide direction for thie teacher durinp thie. instructional process 
and a mor^ appropriate base Tor evaluatlnr a 'person *s tcachinc 
perf orma.nce^ ^ " . 



found th'it:' faculty ten-l'to 07e?v'\^te therrselves^ with respect t(\ 
th^ir overall effectiveness. At the sajne time^ because they have 
received no training on how to fdcus on the really relevant aspects 
of their Instruction, they arit-lolr!e superficialities such As 
appearan'-je , mannerisms '^nd voice tone — none -of which bear any 
significant relationship to eitiier studeht s^Jt is f act 1 on or 
achiever^ent . Relationships investigated between self^-evaluation 
ratinp;s and other criteria', s^c^i as student ratings and measures^ 
of studemt ^ain, have been found to.be ner^lirible. 

3. v>^^t udent ratinr ?. VJithout quecriiofi, the evaluation measure 
most widely employed today is the uhiqul^tous student rating ques- 
tionnaire. '£.*tudent ratlnrrs do^ have some.iierit. They are relative- 
ly easy to collect, provide a comnarat'i vely large data base,, and ' 
even more impo^rtant, provide a means for student ifjput into the 
educational ' sy.step/ (however superficial that may be). 

That they are valid and reliable is supported by the research. 
But so^ is the fact' that tHoy are affected by such favors as class 
size, academic discipline, coujp^se content and faculty experience> • 
To ma*:e promotion and tenure decisions on 'tHe basis* of student 
qpihion would result^ in serious injustices to the teachers of large 
classes or required 'courses In "tourch'' discipline^. 1 ven r\Grre 
serious i's the fact that tfiere JLs simply no evidence" to sup^feVt 
that student ratings are effective aids for improvinr teachinp;. 

• To^discover po st h oc that students felt you didn^t "stimulate 
them '^o hifh ^intellectual leffort'^ fives little constructive feed- . 
back which can help you ^detect why you di.dn^t and how you could do 



so In the future. On the other hand,' finding that yo'p "encouraged 

• • - ~ . . < / ' 

students to express themselves freely and* openly" or "spoke with 

•expressiveness and variety In your tpne of' voice" jnight lead y6u 

to ^oin Actor 's' Equity, but tells you nothing .abouty/^iether the 

^students ^learned as a result of your vibrant tones. * ^ 

^'Qonsumer^satisfaction is tfertainly^ an important* .consideration 

in any evaluation scheme, and student ratings are reliable and 

•vaird indicators of students*' feelings of satisfaction, feut-, as . 

the Dr. Fox exj^eriments amply demonstrated, student satisfaction 



identi ratings in an evaluation ^system"^ 
iliti 'tn mind. . ' • 
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with learning represents little more^than the illusion of having 
learned. Inclusion of studei 
jnust be done with this rea] 

- ^. Crijberion-refeVehced measurement , Measuring change irl 
student behavlpr toward, explicitlydefined educational objectives 
is a relatively nevf and most piromisiijg approach to faculty eval- 
uation. i Commonly referred to ^ criterion-ref erenced measurement, 
this system focuses on the 'knowledge , skills or attitudes studenjts 

have acquii^ed as a result of their instruction . Based on the assumpti*on 

* r , 

that effective teaching mean^ promoting changes in student learn- 

• . r *• 

ingv criteri<5ii-ref erenced evaluation directly measures the teacher's 
impact on student learning, * * * » , • * • - 

Instead of identifying a student's relative status in the class 
(as in the traditional norm-rf ferenced measurement s'ystem), atten- ' • 
tion is focused on -the individual student and hiS" changes in behav- 
ior — In other word^s , changes in his learning. The precise bouh- • » 
daries of the behavior to be assessed are ^Jefined^and criteria for 
Judging the adequacy of the student -responses aije. identified prior 



.to init#ictibn. Thu-^, an -nbsolute rather than a i'eJative Gtandard 

is Used to rrueasupe learning;. . " -n . . ^ 

« * 

-Criterion levels can be .est-jblished a^t any point durinr ^ . 

instruction v/herc infornat i/)n cnnceraln^ tlie tevel of an indl'*^ 

* • . • 

vidUal's rer fornance^ is derrired. The term ''criterion' does'not ' ■ 

necessarily ref>r to. final, ?nd-or-the-cour;se^ behavior . The^.poiMt 

is thfit sp^^ciflc bejnaviors and attitudes innJied at each le,vel of *' 

proficiency are Identified and used to describe the specific tasks 

a student must be ^arable of perforininr. . - j. 

^ One of -the chief pr6ble'^r> -./ith uslnr cri terio,n-r*ef erenced 

measTJrement , or It- r-rowt.^ (-^onlract plans) foir ev-lluatinr' 

, teach.inr, ir. ihrat since different *teacherr set- different coals 

within the^same dlscirllne, and often, even v:thin the sane cours?, 

it is difficult to cofnpare the relative •|:#of J ciency of teachers who 

are pursuinr different goals vdth di f f ererit- r.rouns of -r^t udents The 

adoption of such a procecs also requires the "education of faculty ' 

ar\d admi-nir-trat'or 3 so that the'v both knov; ht^*v to define explicit 

teachinc roalr:, evalijate the v/o^th of th^ ro-^ls, and to construct 

the difficult, but absolTitely necessary, appropriate crjtei^lon- 

referenced tests^. Such -an educational prorran^'ls both e>|pensive 

and time consunin^: ' , ^ / . 

The arP/antareS oC «tbe svsterr to rv mind, hov/ew^r , outv/ei^h ' 

its disudvantares^. Whi3^. the process ic dlffi(?ult and tine^-'con- 

suminc, the payoff is v^ell v;orth the effort. First of al-1, by 

defininr sp^lciflc objectives "^prior to instruction the teacher 

can more Jlkely employ relevant instructional materials that will 

^ , # 

help stuflenls attain the objectives. Tne hrocess of specifically 

defininr ot .loci 1 verj-^forces Instructors tp care.fully think th^ounh 

exa^tly^ what thpy really want to teach, and It Is tfieK.tnuch more 
/ S • 



■ ■ V ' . * . ^ 

likely tint they v/ill he^v.l for ^ruily worthwhile foal^s.' Secondly, ^ - 
'students' prop^ress tov/ard '"at tal ninent of the objectives can be ^ 
continuously monitored so that Instruction can be^ |:ont inuously im- 
proved while tne course is still, in prorress, not after t;he fact. 
E^riall y, the criterion-referenced measurement system provides fo^ 
a f*airer '^valuat^ion of. a faculty member's competence fio A TEACHER. 
There is a mass of evi dence-. th^'Vt no one style or method of teachinr 
produces superior learning. The 'cri ter ion-referenced measur^ement 
system takes this well-established but often forg'otten fact- into 

consideration. The faculty merber v;hose- students make procE^ss. to- 

/ " • ^ ' , 

'V/ard and aci^ievp educational c-oals is*an effective teacher repard- 

\ — 

^ r ^ 

less of the r^etho^ls he uses. 

* • 

The evaluation of teachinr is a fa^ too serious and compricated 
process to .be^ iased solely" on the personal assessment of administra- 
tors, the judrment of visitinr peers, an ?fxamination of course syl- 
labi or teachinr irethods^ or especially student opinions. Each of 
.these is userul, but none is sufficient by itself. They are not 
even that 'valuable in total. V^e must recornize the deficiencies in 
^these traditional approaches,* continue to explore ctnd refine 
1^ (^irect :r»easures of learner crov/th, and, until;t_he p^rfec-t evaluatix)n 

system is found, if ever, opt for multinle indicators of assessing 
^'faculty "performance. 

V/ith this last stateriient in mind, let me now describe a^each- 
ing evaluation profram that my associate, Glenn tJyre, and I have 
recommended be implemented at the UCLA School of Dentistry next 
year. The proposed^ desirn of the program is based- on many of the 
ideas coatained in thi-s f^aner ^ and t^akes Into consic^pfction the 

r 



:ips m 
^ilfl|^e 



ad^antares and d isad vantares' of the various data fomf previously 

>a ^ - - ---- 

described. | ' * 9 



THK UCLA SCHOOL OF PEMTISTR-Y TEACHKR KVALUATIOfJ PR0GRAM 



trst , let me sA[/ that the cllriate In rlrJitl For 'the 



V 



last 'tv/o"veirB , . v/e }^ave* vorke^i \/itn the aflmi rii ntrators and * 
faculty^of the School of Denti-trv* to- establish vthe foundation 
for an onfoinr;, systematic and .coiTiprehensiv^. prcfram of teach- 
inr and -.urricular evaluation.^ V/ith almost complete- and direct 
in^^olvement of the entire faculty and a major portion of the 
student- I ody , explicit, neasuVnl le ro^l were defised for the 



School and Its 1 ' currl cul r^'^'^tlon?. In the process- of- . 
clarifvlrr tli^se ^oals, fax:ulty sav; the noed to become .?qore 
explicit about their ov/i^'^ourse objectives — nak'inr sure that 
what they v;ere teachinr in class corresponded to and contributed^ 
to the larger and more long-range "hool f^oals. Thi's process. led 
to a renev;ed fo^is on t^^achlnf^^, ^nd* an excitement v/as fenerated 
that 'rrsul ted in the faculty aj^j^cinj;; for faculty development 
.v/orkshorc, on everythlnr from the evaluation of instruction to 
the development of a self-naced instructional program. 

A comprehensive education profra^^ f^'^r the faculty v;as thus 
developed at their request v;Mch Incluried teaching thein how to 
specify 'Instructional bhjer'tlve3, to annlyze t!he instructional 



*A more ^comprehensive description of tl^e renewal project at >the 
UCLA 'School of Dentistry can be, obtained by wrltirir to the 
Evaluation] -^and Trnlnlnr. Institute, 11110 Ohio Avenue, r.uitc H?^2 , 
Los Anceles, California 900^5 
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tasks implicit in tlie tnr^^et rc^.lr^, to iticorporate into their ^ 
instruction emrirl ] ly r^r1 veri nstruct 1 ona] practices that 
increase r,tud<^pl- n '^han^^^^r, of --^t, tai nl'rr objortives, to construct 
valid and reliai^le criterion t.c-,tr, anc^ cri ter] on ^ checks to monitor 
sflfcflentG* pro^-ycns tov/ard thn otje^tiven and hov? to nioacure 
student achievenenl and/ at t itudes at the end of the- course.. In 
sum, \aculty v;pre taufht the 'criterion-ref erenc'ed ' system of in- 
struct^n and measurement. 

C'urreritlv, f:iC'ilty ar<-: wc^f^inr with us on .the development 
of a oornprehehsi ve* M'jPGtlQnnair'e which v/ill he used to survey 



recent r^aduates of th^- Tchnol to determine tlieir attitudes 
and behaviors in their practices as they relate, to the f^oals 
which v;ere established. Ih^ this rrvnnrer , the reals tHemselves 
v/ill .be evaluated to determine their relevance to the world 
of the practlcJnr dentjr.t, nnd thus., their v/orth and viability 
for futyre rraduates. '^he qUrf4dulum v;ill be-reyi'sed accord-- - 
innly, Jf necessary, and 'the |vcle of evaluation and chan£:e 
will continue. • . , 

It should be clear that this proup of faculty is ready. for 
^ prorrarr of nyGteinatic teWnhinr evaluation, and their cooperation 
is Insurr^d . They' have learned the rubrics of criterion-referenced 



measurefnent and have used the system In a\ variety of ways . As 

far as their Instructlof* Is concerned, they are cdirimltted to the 

measurement of outcomes as Indicators of stucjent learning and the 

logical next step for them j.s to Implement an outcomes-based 

^eacher evaluation syslem. 
« • 

Colleague visitation and student ratings will continue to be 

used, but colleague visits will be Aiade in ah Informal basis only, 

at the discretion of /the faculty member, and will n6t be Included 

In the evaluation profile. -^Student ratings wlii be used as Indl-- 

cators of satisfactlpn only, and lnclude(j with them will be' data 

concerning the size of the class, 'the nature of the course and 

whather or not the course Is required — factors that have been 

shown t„o Influen-ce student r^lngs. The main thrbst of th^ evalua 

tlon profile will be a combination of concrete evidence of direct 

teacNhing effectiveness within each -course . 

• The evaluation program will begin before classes start In 
. ' ' • ' - - / * , 

September with a meeting of each faculty ''meITlbe^ and the section 

chairman. At this time, n;he objectives foV each course taught by 
the facultiy member! will be reviewed, as well as the evidence that 
will' be collected from students to determine their degree of at- 
tainment . Levels of performance will be established for both ^ 
individual students and the class, and the conditions upder which 
the students' behavior^ are to be demonstrated will be specified. 
Although each f^cylty^ member will 'be expected to continuously 
monitor class progress throughout the course, for ^he first^* 
year the faculty member and section chairman will fiot meet 

again until the end of the quarter- (unless the faculty member 



requests such a meeting), at which ,time they^wdll review th^ • ♦ 
level df class achievemervt, indiviflual stud^nta a^chlevement arid 
the objectives themselves. As a result ' ofithis analysis, obj 6c-' • 

^tive^^ay be reformulated for the next ..alass' anp €Xi>ectatiohs 
regarding student and cliss'^levels o/ acii^ey^^meyit' piay be reylsed/ 

-At the end of 'the academic year,^ the section chai/man will 
meet with faculty both individually and alS a group to dJfetier^ine th^ 
overall efficacy and interrelationship of the course objectives 
and the ^ener?il level of student achievement 1 By reviewing an 
individual's course evaluations' from three quarters , an assessment. 
willLe made of each faculty .member * s ability* to promote student, 
learning generally, which ob'Jectivers^ proved too dif-ficult-to ac- 
complish/ what methods proved eTfective or ineffective, and^BO on. 

' Student satlsf iicti^on based on studefit rating forms administered 4 
to each class each quarter Will alSo be reviewed and areas of 
dissatisfacti^ analyzed. It should be rtoted, however, that the 
purpose and emphasis of this 'program Is d jlagnostic ' ;^ 

to assess each faculty member's purrh^^ level of teaching ef fdsc- 
tiveness, identify problem areas, and establis^h the foundation for , 
improvement . - , , , 

An integral and parallel part of the' evaluation program will 
necessarily be an on-going instructional development prograki to 

- which faculty can come for ,help any time they wish. We believe 
that it is impractical 'and foolhardy to establisl^ a; teaching eval- , ^ 

uation program without an '^accompanying improvement program* ^ 

/ ' ' * . ' ^ 

By ha-ving • aval-lable to facility an instruictional im- 

provement program congruent with the ingredients ofj the evaluation 



program, they will receive the kind of help^that wijLl enable them 

if * \ ' . I 

to maximize attainment of their cbntracted responsibili'ty . Nothing 



sxicce^'ir, like ?uocess J-tself. ^Thur.» fcr the next .^ear^.'^nd 

"ppssibly even the -second year, the focus/of ^the ev^ilatJon . * • 

.program will be pn teaching improvement and not on promotion 

tenure. Once the faculty have ha<i a chance_to improve their- 

tea'chlnr -ef rectlvenes's, .to demonstrate their ability 'to brinr: 

aboufe. student learrfinp , • and to practice the -'process of collect-- 

-inc outc^ome data, the end-of-year- chairman/ racul't;y mee'lbine should 

ultimately r*estil,t, in an eval^uation profile that can- be used . 

(along with 'traditional^ e^^idence of schoiarJLy "rlsearc+i and com- 

munity service) in -arrivlnr at peappointm^ent , promotion and . . 

tenu're decisions-. Teaching wlH ^no lonrer l^e ,the ugly step- . 

sistert of the rroup.- ^ ' ' ,^ . . ^' 

The experimental teaching evaluation program' I lia^^ Just^ de-^ 

scribed (as well as our entire change and renewal project at the' 

School of Dentistry) is based on three major assumptions: • 1) that 

the goal of curricular tlevelopment and organizational change is 

the improvement of teaching and learning; 2). thai: any really mean- 

ingful changes , in- the curr'ic:urum^ and , ultimately, improvement in 

th«-^e'aching/learning process- must be based upon a rigorous evalua- 

tion pi?pgram; and 3-) the focus of such an, evaluation program must 

♦ * 

be on outcomes — outcomes in terms of student- achievement arid 
satisfactlron; faculty developmeYit $tnd satisfaction; the responsive- 
ness of the curriculum; and f inally^outcomes Xn terms T)f the total 
school and organizational environment. , ^ , 

These are the conditions necessary for a. viable faculty 
•^valuation program. Accountability means 'more than Just instruc- 
ticfhal accountability, arid Institutional cost-effectiveness. It 
•means the ent'ire system af higher education is accountable for^'^ 



providing stud^ts with an effective, responsive and satisfying 
edxicatfon. It means tl^t the goals of each institution* mu.st be * 
defined 'in order to determine what constitutes a "good" education 
for, the students at that institution, 'It means that' institutix^nal 
changes' must be m^de in Ithe environment and particularly the re- 

ward structtfre so tha^; faculty member^ who are truly effective 

< ' . ' ^ ' . • 

teachers are recognized artd encouraged in their efforts • It 

, » * » . ■ ' ' ' /f-^ 

means that Institutions must mal^e a serious i^^'^itment to Improv- 

ing Instruction and evaluating teaching by providing substantial 

financial resources and personnel support, I am convinced that 

unless -colleges and, universities today make such a commitmerit, 

+iigher education may never recover 'its former importance and respect 

in soQiety\ That would be a sad ending iVideed, 
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